Few-Shot Composition Learning for Image Retrieval with Prompt Tuning

We study the problem of composition learning for image retrieval, which we learn to retrieve target images with search queries in form a reference and modification text that describes desired modifications image. Existing models retrieval are generally built large-scale datasets, demanding extensive training samples, i.e., query-target pairs, as supervision, restricts their application scenario few-shot only few pairs available. Recently, prompt tuning frozen pretrained language has shown remarkable performance when amount data is limited. Inspired by this, propose mechanism CLIP model task retrieval. Specifically, regard representation trainable visual prompt, prefixed embedding sequence. One challenge efficiently train samples. To deal this issue, further self-upervised auxiliary via ensuring can itself no information given from text, facilitates while not requiring additional annotations pairs. Experiments on multiple benchmarks show our proposed yield superior trained